This papers investigates theoretically and numerically why the recent BinaryConnect (BC) works better in comparison to more traditional rounding schemes, such as Stochastic Rounding (SR). It proves that for convex functions, (the continuous weights in) BC can converge to the global minimum, while SR methods fair less well. Also, it is proven that, below a certain value, learning in SR is unaffected by decreasing the learning rate, except that the learning process is slowed down. The paper is, to the best of my understanding: 1) Clear, modulo the issues below. Specifically, I think it helps clarify why is it so hard to train with SR over BC (it would be extremely useful if one could use SR, since then there would be any need to store the full precision weights during training).